Outline

  1. Introduction to Bayesian inference
  2. The BayesFactor package
  3. \(t\)-Test
  4. ANOVA
  5. Linear model setup

Introduction to Bayesian inference

Example

  • Effect of media stories on attitudes towards Syrian refugees.
  • IV: Sympathetic media story about refugees or control stofy about food trends.
  • DV: Participants’ favorability ratings towards refugees.
  • Also recorded: Political affiliation of the participants.



Example from Rouder, Haaf, & Aust (2018)

Usual approach

  • Conduct a \(t\)-test to assess the effect of media stories on attitudes towards Syrian refugees.
  • Conduct a 2x2 ANOVA to assess the effects of both media stories and political affiliation as well as their interaction.
  • Neither necessarily match the research questions associated with the study.
  • Baysian model comparison allows to better match inference and theoretical questions.
  • But you can also do the regular stuff.

Three main theoretical positions

  1. The plight of refugees generates an empathic response which results in an increase in favorability for conservative and liberals alike.
  2. Confirmation bias (Tversky & Kahneman, 1974): People ignore media stories incompatible with their world view. Therefore, only liberals are affected.
  3. Belief polarization (Cook & Lewandowsky, 2016): Liberals see refugees more favorably after viewing positive stories while conservatives might actually see them more negatively.

Testing position 1

  • We need models

Testing position 1

  • We need models
  • \({\cal M}_1:\) The effect of media stories on attitudes is positive.


Testing position 1

  • We need models
  • \({\cal M}_1:\) The effect of media stories on attitudes is positive.
  • \({\cal M}_0:\) There is no effect of media stories on attitudes.

Testing position 1

  • The prior distribution indicates the plausibility of effects before the data have been collected.
  • Because we make this fine-grained specification, predictions are now possible.

Testing position 1

Evidence through predictions

  • In Bayesian statistics, evidence for models reflects how well they predict data.
  • Prior is on true values.
  • Predictions are on data.

Testing position 1

Evidence through predictions

  • In Bayesian statistics, evidence for models reflects how well they predict data.

Testing position 1

Evidence through predictions

  • In Bayesian statistics, evidence for models reflects how well they predict data.

Testing position 1

Evidence through predictions

  • In Bayesian statistics, evidence for models reflects how well they predict data.

Testing position 1

Evidence through predictions

  • In Bayesian statistics, evidence for models reflects how well they predict data.

Bayesian model comparison in practice

The BayesFactor package

  • Based on Rouder, Speckman, Sun, Morey, & Iverson (2009) (\(t\)-test), Rouder et al. (2012) (ANOVA), and Rouder & Morey (2012) (regression), total of > 4000 citations.
  • Same setup is implemented in JASP and SPSS.
  • Provides a convenient interface for commonly used models.
  • Bayesian versions of typical frequentist tests using Bayes factor.


The BayesFactor package

Functionality

  • General linear models (including linear mixed effects models): generalTestBF, lmBF
  • Linear regression: regressionBF, lmBF
  • Linear correlation: correlationBF
  • \(t\)-tests: ttestBF
  • Meta-analytic \(t\)-tests: meta.ttestBF
  • ANOVA: anovaBF, lmBF
  • Contingency tables: contingencyTableBF
  • Single proportions: proportionBF

The BayesFactor package

Functionality

  • General linear models (including linear mixed effects models): generalTestBF, lmBF
  • Linear regression: regressionBF, lmBF
  • Linear correlation: correlationBF
  • \(t\)-tests: ttestBF
  • Meta-analytic \(t\)-tests: meta.ttestBF
  • ANOVA: anovaBF, lmBF
  • Contingency tables: contingencyTableBF
  • Single proportions: proportionBF

\(t\)-Test

\(t\)-Test

Can be used to test our position 1 .

Equation setup

\(t\)-Test using BayesFactor

BayesFactor::ttestBF(formula = attitude ~ story
                     , data = immigration_study)

Prior settings

  • \({\cal M}_1:\) The effect of media stories on attitudes is positive.
  • \({\cal M}_0:\) There is no effect of media stories on attitudes. >- Attitudes might be normally distributed: \(Y_{ij} \sim \mbox{Normal}(\mu + x_j \theta, \sigma^2)\). >- For model \({\cal M}_0\) \(\theta = 0\). >- For model \({\cal M}_1\) we need a prior on mu: \(\theta\) >- Prior on \(\sigma^2\) and \(\mu\)?

Three reasons for using informed priors

1. You gotta know something

You are…


  • A researcher in psychology.
  • Working in their field of expertise.
  • Standing on the shoulders of previous generations of researchers.




2. Informed priors \(\rightarrow\) informative tests

Obtain strong evidence by increasing model discriminability.

Stefan, Gronau, Schönbrodt, & Wagenmakers (2019)

3. Informed priors ensure identifiability

  • Complex models often suffer from identifiability issues.

  • Informed priors introduce probabilistic constraints.

The blessing of default priors

  • BayesFactor package in R and JASP use default priors for the t-test called JZS prior.
  • Prior structure is consistent.
  • User-defined prior settings are on the scale of the effect size:
  • \(\delta = \theta / \sigma\) (think Cohen’s \(d\)).

The blessing of default priors

  • BayesFactor package in R and JASP use default priors for the t-test called JZS prior.
  • Prior structure is consistent.
  • User-defined prior settings are on the scale of the effect size.
BayesFactor::ttestBF(formula = attitude ~ story
                     , data = immigration_study
                     , rscale = 1 / sqrt(2))

The blessing of default priors

BayesFactor::ttestBF(formula = attitude ~ story
                     , data = immigration_study
                     , rscale = 1 / sqrt(2))

The blessing of default priors

par(mar = c(3,3,.5,.5), mgp = c(2,.7,0))
x <- seq(-3, 3, .01)
y <- dcauchy(x, 0, 1/sqrt(2))
plot(x, y, type = "l", lwd = 2
     , ylab = "Density", xlab = "Effect Size"
     , col = "darkblue", ylim = c(0, .5))
lines(x, dnorm(x), col = adjustcolor(1, .5), lty = 2, lwd = 2)
abline(v = c(-1, 0, 1), lwd = 0.5, col = adjustcolor(1, alpha.f = 0.3))

Prior prediction

##         0%        25%        50%        75%       100% 
## -755409.12      -1.24      -0.01       1.21  109714.01

The blessing of default priors

BayesFactor::ttestBF(formula = attitude ~ story
                     , data = immigration_study
                     , rscale = 1 / 4)

Your turn!

  1. Plot the default prior on effect size.
  2. Adjust the scale until it matches your expectations of the outcome of the study.
  3. Conduct a \(t\)-test using your setting.
  4. Do you think any two people in the workshop get the same result?


What was your scale setting?

Put your setting in the zoom chat!



What was your scale setting?

\(t\)-Test using BayesFactor

Output

BayesFactor::ttestBF(formula = attitude ~ story
                     , data = immigration_study
                     , rscale = 1 / 2)

Back to media stories about refugees

The plight of refugees generates an empathic response which results in an increase in favorability for conservative and liberals alike.

  • Does the \(t\)-test as we used it address this statement?

Estimating the effect

est.attitude <- BayesFactor::ttestBF(formula = attitude ~ story
                     , data = immigration_study, rscale = 1 / 2
                     , posterior = T, iterations = 20000)
plot(est.attitude[,"beta (Neutral - Refugee plight)"])

Estimating the effect

Back to media stories about refugees

The plight of refugees generates an empathic response which results in an increase in favorability for conservative and liberals alike.

  • Does the \(t\)-test as we used it address this statement?
  • We need an ordinal constraint.
  • Ordinal constraint: Specification of the direction of an effect.

One-sided tests with BayesFactor

  • One-sided \(t\)-tests are easy with BayesFactor.

One-sided tests with BayesFactor

  • One-sided \(t\)-tests are easy with BayesFactor.
BayesFactor::ttestBF(formula = attitude ~ story
                     , data = immigration_study
                     , rscale = 1 / 2
                     , nullInterval = c(-Inf, 0))
## Bayes factor analysis
## --------------
## [1] Alt., r=0.5 -Inf<d<0    : 4.196947   ±0%
## [2] Alt., r=0.5 !(-Inf<d<0) : 0.08881904 ±0%
## 
## Against denominator:
##   Null, mu1-mu2 = 0 
## ---
## Bayes factor type: BFindepSample, JZS

Your turn!

  1. New data set, this time real data.
  2. Run a Bayesian t-test, try out ordinal constraints and estimation, maybe even prior prediction.
  3. As questions while I am here!
SourceURL <- "https://raw.githubusercontent.com/PerceptionCognitionLab/data0/master
/contexteffects/FlankerStroopSimon/cleaning.R"
devtools::source_url(SourceURL)

Your turn!

library(tidyverse)

stroop.agg <- stroop %>%
  group_by(ID, congruency) %>%
  summarize(mrt = mean(RT)) %>%
  spread(congruency, mrt)

Analysis of variance

ANOVA

  • Based on Rouder et al. (2012).
  • Between-subject, within-subject and mixed designs.
  • Random effects.
  • Perhaps the most thought out functionlity in BayesFactor.

ANOVA models

Thank you!


Cook, J., & Lewandowsky, S. (2016). Rational irrationality: Modeling climate change belief polarization using Bayesian networks. Topics in Cognitive Science, 8(1), 160–179. Retrieved from https://dx.doi.org/10.1111/tops.12186

Rouder, J. N., Haaf, J. M., & Aust, F. (2018). From theories to models to predictions: A Bayesian model comparison approach. Communication Monographs, 85, 41–56. Retrieved from https://doi.org/10.1080/03637751.2017.1394581

Rouder, J. N., & Morey, R. D. (2012). Default Bayes factors for model selection in regression. Multivariate Behavioral Research, 47, 877–903. Retrieved from http://dx.doi.org/10.1080/00273171.2012.734737

Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012). Default Bayes factors for ANOVA designs. Journal of Mathematical Psychology, 56, 356–374. Retrieved from http://dx.doi.org/10.1016/j.jmp.2012.08.001

Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian \(t\)-tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin and Review, 16, 225–237. Retrieved from http://dx.doi.org/10.3758/PBR.16.2.225

Stefan, A. M., Gronau, Q. F., Schönbrodt, F. D., & Wagenmakers, E.-J. (2019). A tutorial on Bayes Factor Design Analysis using an informed prior. Behavior Research Methods, 51(3), 1042–1058. doi:10.3758/s13428-018-01189-8

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131. Retrieved from https://doi.org/10.1126/science.185.4157.1124